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I Introduction 


In the last years great efforts have been undertaken to unravel the mystery of quantum chaos, 
i. e., the search for the fingerprints of classical chaos left on the corresponding quantum systems. 
The main focus has been on two-dimensional billiard systems, which constitute the simplest 
class of systems being strongly chaotic. The chaotic behaviour is classically imprinted in the 
structure of phase space where topological differences occur between Hamiltonian systems with 
two degrees of freedom and more than two degrees, e.g., the Arnold diffusion in KAM theory 
is only possible in three or more degrees of freedom. Thus it is about time to investigate also 
the quantum mechanics of chaotic systems with more than two degrees of freedom, since the 
different phase space structures semiclassically influence quantal energy levels as well as the 
eigenstates. It was Gutzwiller [1], who first suggested to study three-dimensional hyperbolic 
billiards, being embedded in a space of constant negative curvature, as ideal models for quantum 
chaotic systems in higher dimensions. 

Jacobi’s equation for the geodesic deviation shows that in the case of constant negative 
curvature neighbouring geodesics always diverge at an exponential rate thus yielding one of 
the two ingredients of chaotic systems. The other requirement is a finite phase space which 
is attained by the finite configuration space of the billiard. Before we concentrate on the 
three-dimensional case we would like to recall some facts about the two-dimensional case. In 
the two-dimensional hyperbolic space with constant curvature, polygons are cut out from the 
hyperbolic plane such that their geodesic edges can be identified in pairs by imposing periodic 
boundary conditions. This can also be viewed as a tessellation of the whole hyperbolic plane. 
In this way a fundamental cell is obtained on which the dynamics of a free particle can be 
investigated. The topology of the surface is determined by the orientation and Euler’s invariant 
w = v — e + /, where u, e and / denote the number of vertices, edges and faces, respectively. For 
a given w a Teichmiiller space exists which parameterizes surfaces having the same topology. 
The same procedure in three-dimensional hyperbolic space, i.e., cutting out polyhedra, does 
not yield a parametric family of spaces with the same topology. One instead obtains a variety 
of different topological species which cannot be described by a simple measure like Euler’s 
invariant in two dimensions. This difference is caused by the many conditions one has to fulfill 
in order to get a fundamental cell which tessellates the three-dimensional hyperbolic space. To 
tessellate the hyperbolic space with a polyhedron one has to require that the dihedral angles 
around an edge add up to 2ir and the solid angles forming a vertex add up to 47T, as will be 
explained later. In addition to these conditions one has to obtain a polyhedron whose faces 
can be identified in pairs. These restrictions are the reason for having only “discrete” models 
in the three-dimensional case. To choose a simple model for the study of quantum chaos we 
concentrate on a pentahedron being symmetric along an intersection plane, see figure 1. (To 
identify faces in pairs view it as a hexahedron with one dihedral angle of ir.) Desymmetrization 
of the pentahedron leads to a tetrahedron, the most fundamental three-dimensional object. 
Instead of the periodic boundary conditions of the pentahedron one has Dirichlet and Neumann 
boundary conditions in the desymmetrized system, which facilitates the numerical computation 
of quantal levels. The tetrahedron itself shows a rotational symmetry (see end of section II). 
If one is interested in spectral statistics it is important to desymmetrize the considered system 
totally. However, it is easier to illustrate the connection between geometric quantities and 
quantum energy levels, if we keep the geometry simple. 

The quantum mechanics of a chaotic system can only be understood in connection with the 
classical counterpart. Geometric quantities of the classical system contain all information about 
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the spectrum of quantal levels via an exact trace formula that takes over the role of Gutzwiller’s 
semiclassical trace formula [2]. The trace formula can be used as a quantization condition for 
the energy levels as well as for a statistical description of the number variance E 2 (T) or the 
spectral rigidity A 3 (L). An important contribution in the trace formula is due to the classical 
periodic orbits and the geometric quantities attached to them. These geometric quantities 
arise by the linearization of the motion along the periodic orbit thus describing its infinitesimal 
neighbourhood. From three dimensions on there exists an additional type of periodic orbit, 
the so-called loxodromic orbit. The neighbourhood of such a periodic orbit is rotated after one 
traversal, a property being impossible in two dimensions. 

This paper is organized as follows. The next section provides together with the appendix a 
derivation of the trace formula for general co-compact lattices of three-dimensional hyperbolic 
space. The hyperbolic tetrahedron is a fundamental cell of such a hyperbolic lattice being 
generated by the reflections at the faces of the tetrahedron. Until now the trace formula was 
only available for lattice groups without reflections, i. e., for polyhedra with periodic boundary 
conditions. Section III describes in more detail the structure of hyperbolic reflection groups 
with special emphasis on lattices having a tetrahedral fundamental cell. An average law for 
the spectral staircase is derived from the trace formula and applied to the considered system. 
Section IV deals with the numerical computation of the periodic orbits, which are used for the 
calculation of quantal levels via the trace formula in section V, where the spectral staircase and 
the “cosine-quantization” are discussed. The results obtained by the trace formula are compared 
with numerical computations of the quantal levels using the boundary element method. 


II Geometry and trace formulae 

There exist several models for hyperbolic space, see e. g. [3]. For our considerations we prefer 
the upper half space 

(1) #3 = {(x 1 ,x 2 ,x 3 ) G R 3 I i 3 > o}, 


equipped with the Riemannian metric 

2 dx\ + dx 2 + dx\ 

ds =k - 2 -• 

x 3 

The curvature equals — 1/k at every point of T) 3 ; we set k = 1. The main advantage of this model 
is that the action of the group of isometries on T) 3 (i.e., the distance preserving bijections on T) 3 ; 
we will denote them by IsoT) 3 ) has a simple representation by linear fractional transformations: 
Set x = x 1 + x 2 i + x 3 j, with the quaternions i and j defined by the relations i 2 = j 2 = —1 and 
ij + ji = 0, plus the property that i and j commute with every real number. The inverse of a 
quaternion q = qi + q 2 i + q 3 j + q 4 ij ^ 0 is then given by q~ x = \q\~ 2 (qi — q 2 i — q 3 j — <jqij), where 
I d 1 2 = di + dk + di + dl■ ^ can shown that every orientation preserving isometry / of T) 3 has 
a representation 


( 3 ) 


f(x) = (ax + b)(cx + d) 1 , where 


“ ) e SL(2,C), 


while for the orientation reversing case this matrix should be contained in the coset SL(2,C)j, 
i. e., the matrix elements are of the form zj, z £ C. It can easily be deduced that the group of 
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orientation preserving isometries Iso + X) 3 is isomorphic to PSL(2,C) = SL(2, C)/{±1} and the 
class of orientation reversing isometries is isomorphic to PSL(2,C)j. We say that an isometry 
f belongs to a matrix F £ SL(2,C) U SL(2,C)j. 

The isometries of X) 3 can be classihed the following way [4]: 


/ is called ... , 
plane reflection 
elliptic 

inverse elliptic 
parabolic 
inverse parabolic 
hyperbolic 
inverse hyperbolic 


if F is SL(2, C)-conjugate to . . . 


± 

± 

± 

± 

± 

± 

± 


1 0 
0 1 


J 


e FF o \ 

0 e-W 2 ) ’ 

0 ie^ 2 \ . 

ie-i^/ 2 0 ) J ’ 

1 M 

0 l) 

1, !)i 

e ;/2+i0/2 

0 


£ (0, tt] 

<f> € (°W] 


e '/ 2 0 

0 e -'/ 2 l ] 


— l/ 2 —i(f )/2 y ’ / > 0, <j) £ [0, 2?r) 

/ > 0 


We call l = If the length and <j> = <ff the phase of the transformation. In case of elliptic elements 
the phase <f corresponds to the rotation angle. Hyperbolic elements are called loxodromic , if 

(p 0. 

Take a discrete subgroup T of Iso X) 3 , and identify all points of X) 3 which can be transformed 
into each other by an element of T. Those points are called Y-equivalent and we put them into 
an equivalence class T(x) = {g(x) \ g £ T} with x £ T) 3 . The set of those classes is the 
hyperbolic three-orbifold T) 3 /r = {r(a:) | x £ T) 3 }. 

To visualize the orbifold we take one representative from each class such that the set of 
all representatives yields a simply connected set in T) 3 , called fundamental cell Ft- If the 
fundamental cell is of finite volume, we call T a hyperbolic crystallographic group or simply a 
hyperbolic lattice. If the closure of the fundamental cell is compact, T is called a co-compact 
lattice. A hyperbolic lattice is co-compact if and only if T contains no parabolic element - 
otherwise the hxpoint of the parabolic transformation, which is at infinity, would be contained 
in the fundamental cell. Discrete subgroups of Iso + T) 3 are called Kleinian groups. 

The dynamics of a classical point particle moving freely in the space T) 3 /r is strongly chaotic, 
if the volume of T) 3 /r is finite. The dynamics reflect the properties of an Anosov system [5], to 
be precise. (This is true for any closed hyperbolic orbifold in any dimension.) 

Turning to quantum mechanics we study the stationary Schrodinger equation (in units 

h = 2 m = 1) 

(4) - Kfl{x) = Efl(x), £ L 2 (fVT, X), 
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A denotes the Laplace-Beltrami operator 

2 / d 2 d 2 d 2 \ d 

3 dxl dx 3 J X 3 dx 3 ' 

L 2 (f) 3 /r,x) is the space of all functions which are square integrable on the fundamental cell 
and T-automorphic, i.e., satisfy for all g £ T 

( 6 ) 4>{g{x)) = x(g) ^( x ), 

where y i s any one-dimensional unitary representation of T, so-called character. For example 
take a reflection group - like we will do later on - generated by the reflections gi,. . . } g n at 
faces of a hyperbolic polyhedron, and choose x(di) = — 1 for all those generators, then one will 
obtain Dirichlet boundary conditions on the faces of the polyhedron. Choosing x(di) = 1 yields 
Neumann boundary conditions. 

In the following we will study the eigenvalues of the operator —A on a compact orbifold 
# 3 /r, where the spectrum is discrete, 


(5) 


0 < Ci < E, < • • • . 


Difficulties arising in the non-compact case from the continuous part of the spectrum are of 
pure technical nature. They will cause some additional terms in the trace formula which are 
of minor importance. Therefore let us not lose track and concentrate on compact orbifolds. To 
obtain information about the energy spectrum we look at the trace of the resolvent of equation 

( 4 ), 

OO 

(7) tr [(-A -EX] =Y.(E„ -E)- 1 . 

n=l 

This series diverges, however, as the number of eigenvalues below an energy E is asymptotically 
(E —> oo) proportional to E 3 / 2 according to Weyl’s law, so the energies E n grow just like n 2 / 3 . 
Let us replace the divergent trace by 

OO 

(8) tr [(-A - E )"' - (-A - £')-'] = £ [(£„ - E)~' - (E n - E')" 1 ] , 

n=l 

which is absolutely convergent. Since E' ^ E n is fixed, the poles of the regularized resolvent are 
still at the eigenvalues. The integral kernel of the resolvent is the Green function Gr(x } x 1 ; E) 
of equation (4) which is a coherent superposition of the free Green function G(x, x'; E) on Sj 3 , 

(9) G r (x, x'; E) = J2 x(fl') G(g(x), s'; E), 

ge r 


where 


(10) G(x,x':E) = --—--- expfi pd(x,x')], p = VE — 1. 

v y v y 47 t smh d(x, x’) v y 

d(x,x') denotes the hyperbolic distance between x and x 1 £ Sj 3 , dehned by 


( 11 ) 


cosh d(x, x') = 1 + 


x x f p 
2x 3 x 3 
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It should be noted that the three-dimensional Green function contains only elementary func¬ 
tions, while in two dimensions Legendre functions are involved - a characteristic difference 
between even and odd dimension. 

Finally we are left with the integral 

(12) tr [(—A — if) -1 — (—A — if') -1 ] = j dp(x) [Gr(x, x\ E) — Gr(x, x\ E')\ } 

Ft 


p(x) being the volume element of T) 3 , 


(13) 


dfi(x) = 


dxidx 2 dx s 


The integration is performed in the appendix, the result reads 


(14) 


oo 

£ [ipl -P 2 ) -1 - (pI -p' 2 )~\ 


71 = 1 


VoljEr 

47ri 


ip-p) 


~ £ x{p) Ar t!^ - i p) + fH-i p) - VE - E) - V’(-V)] 


Mr 


167r 

x(p) U 


1 1 


_ y _ 

4ord£ r (p) (1 - cos (j> p ) \yp i p' t 

ellipt. 

, y- xjp) 

{p}r ord ^r(d) 

inv. ellipt. 


[I ip, <Pp) - Hp^p)} 


- £ 


X{r)l 


TO 


exp(ip/ T ) exp(ip'/ T 


l p 


^ 4ord^r('?') (cosh/ T — cos cf> T ) y 

hyperbol. 

X(r)/ T0 /exp(i p l T ) exp(ip'/ T ) 


ip' 


- E - 

r~f 4ord£r(T) sinh / 

{r}r v > 

inv. hyperbol. 


ip 


ip 


for Imp > 1, Imp' > 1. We have substituted the energy variables by momentum variables, 
setting E = p 2 + 1, E' = p' 2 + 1 and E n = p 2 n + 1, where —i p n £ (0,1] for n = 1,. . . , N, if N is 
the number of all so-called “small” eigenvalues E n £ [0,1), and p n > 0 for n > N. 

The sums in (14) extend over T-conjugacy classes 

(15) Mr := {g' \g' = hgh~ x , h £ T} 


of plane reflections, elliptic, inverse elliptic, hyperbolic and inverse hyperbolic transformations, 
denoted by p or r, respectively. V p is a collection of all parts of the fundamental cell, which 
are left invariant by the reflection p or one of its conjugates. r 0 is a transformation with a 
shortest length l To of all hyperbolic and inverse hyperbolic transformations commuting with 
p or r, respectively. ord£r(/) denotes the order of a maximal finite subgroup £r(f) of the 
centralizer Cr(/), which is the subgroup of T that contains all elements commuting with /. 
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ip(x) is the logarithmic derivative of Euler’s gamma function, the function /(p, <f>) is dehned in 
the appendix. 

The distinct contributions to the trace formula can be interpreted in a nice geometric way: 
The first summand depends on the volume of jj 3 /r. The second contribution is connected with 
the orbifold’s surface (if there is any), it is a sum over distinct surface parts belonging to chosen 
boundary conditions. If one chooses a unique boundary condition on the whole surface (e.g. 
Neumann or Dirichlet type), one will obtain a contribution 

surface area r , . , . , 

T- Yfa - Wv “ l P) + N(-ip) -ip{l-ip)- n(-i P )J, 

upper sign for Neumann, lower sign for Dirichlet boundary conditions. The third contribution is 
connected with lengths and dihedral angles of the edges of jj 3 /r, the fourth one with its corner 
angles, in a way that will be explained by example in section III. There is a correspondence 
between closed geodesics of jj 3 /r and conjugacy classes of hyperbolic and inverse hyperbolic 
elements of T. This relation is, however, in general not one-to-one: Let {r}r be a hyperbolic 
or inverse hyperbolic conjugacy class associated with a periodic orbit. Then all conjugacy 
classes of the form {rp} p, p £ £p(t) belong to the same orbit. However, ord^p^) 7 ^ 1 only for 
surface or edge orbits. Periodic orbits for which ord£p(r) = 1 will be called interior orbits. l T 
corresponds to the length of the periodic orbit associated with {r}p. 

Let us compare our periodic-orbit contribution to the trace of the resolvent with that one 
obtained by Gutzwiller’s theory [2], which is given by 


(16) 


Xpo 4o ( exp(i p Ipp) 

2 | det(Mp 0 — l)! 1 / 2 \ i p 


exp(i p' l PO ) \ 

i P' ) ’ 


where Xpo is a phase factor involving the Morse index of the considered periodic orbit PO and 
boundary conditions, / PO its length, and / PPO the length of the associated primitive periodic 
orbit PPO. The monodromy matrix M PO is a linearized map of the four-dimensional surface of 
section (at an arbitrary point P on the periodic orbit) onto itself, which describes the traversal 
of phase space trajectories, starting and ending at points on the surface of section nearby P. 

With every interior orbit we can associate exactly one conjugacy class {r}p, where r is hyper¬ 
bolic or inverse hyperbolic, depending on the periodic orbit being direct or inverse hyperbolic. 
The monodromy matrix is a map M PO : IR 4 — > IR 4 , (Sq P} Sq 2 , hpi, hp 2 )* (—(• .1/. u'</|. Sq 2 , hpi, Sp 2 y. 
Sqi are local position coordinates and Spi local momentum coordinates of a coordinate system 
orthogonal to the periodic orbit at P. A straightforward calculation shows that 


(U) 

det(M PO - 1)| 1/2 


2 (cosh / PO — cos </> PO ) if the periodic orbit is direct hyperbolic 
2 sinh / PO if the periodic orbit is inverse hyperbolic, 


in accordance with trace formula (14). 

This result can easily be generalized to the n-dimensional case, where we obtain for n odd 

j 

| det(Mp 0 — I)] 1 / 2 = 2 fc • DJ(cosh/ PO — cos</y PO ) • (sinh l PO ) k ~ 3 , 

8 = 1 

with k := (n — l)/2, and j phases </y PO , j £ {0,. . . , k}. The considered periodic orbit PO is 
direct hyperbolic, if k — j is even, and inverse hyperbolic otherwise. In the even dimensional 
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case we have 


| det(Mp 0 - 1)| 1/2 


3 

2 k ■ cosh(/ PO /2) • ]^[(cosh/po — cos</y PO ) • (sinh / P o ) fc_,?_1 

8 = 1 
j 

2 k ■ sinh(/ PO /2) • (cosh / PO — cos </y PO ) ■ (sinh / P o ) fc_,?_1 
8 = 1 


with k := n/ 2, and j phases </y PO , j £ {0,... ,k — 1}. The upper (resp. lower) case corresponds 
to a direct hyperbolic orbit, if A; — j — 1 is even (resp. odd), and to an inverse hyperbolic orbit 
otherwise. Let us return to n = 3. 

In the case of surface or edge orbits M PO cannot be determined in a unique way: Let £ be 
the discrete subgroup of 0(2) that leaves the edge (resp. the surface part) pointwise invariant. 
£ identifies points on the surface of section via the action defined by 


(18) ®j(6q 1 ,6q 2 ,6pi,6p2)\ p££, 

where 0 denotes the 2x2 zero matrix. Therefore M PO is a map M PO : IR 4 /£ —s- IR 4 /£, and with 
Mpo there are additional (ord£ — 1) matrices of the type 


(19) 


Mpo 


Mpo 


P o 

0 p 


p £ £, 


which do not differ in their action on IR 4 /£, but have different determinants det (M PO — 1) in 
general. Let us see what contribution instead of (16) is suggested by trace formula (14). 

First it can be shown that there exists a hyperbolic element r associated with the considered 
periodic orbit, such that £t{t) is isomorphic to £. If two elements pi, p 2 of £t{t) are £t{t)~ 
conjugate, then {rpi}r = {t/Gjt- It is an algebraic exercise to show that, if k elements 
pi,. . . , pk of £t{t) are df^-conjugate, then 


( 20 ) 

Thus 

( 21 ) 


ot&.£y{t pi) = ord£r(T)/A, i = 1,... , k. 


Mp ord ^r(rp) 

r fixed 

pe£r( T ) 


ord ^r(7") pe ^( T ) 1 1? 


where the first sum runs over all distinct conjugacy classes {rp}r, which means that the elements 
p G £t{t) are not df^-conjugate to each other, r is fixed and satisfies the conditions described 
above. The contribution of a surface or an edge orbit to (14) is then given by 


( 22 ) - 


X 

t T P}r 
r fixed 
pe£+(r) 


X(TP) l r 


4 ord£r(Tp) (cosh l T — cos <f> Tp ) 


+ £ 


X(TP) 


t T P}r 
r fixed 
p££ p (t) 


4 ord£r(7"p) sinh l T 


ex 


p(i pk 


l p 


exp (ip' lj 
ip' 
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or, written as a sum over all elements of £p(t), 


(23) - 


; 


TO 


4ord£r(V) 


£ 

oG£+(t) 


X(rp) 


COsll l T — COS y Tp 


+ £ 


p££ r (r) 


X(rp) 

sinh L 


exp(ip/ T ) exp(ip'/ T ) 


1 p 


ip' 


where (r) denotes the set of orientation preserving and orientation reversing elements of 
£y(t), respectively. In the case of edge and surface orbits we should therefore replace (16) by 


(24) 


bpo _ Xpo _ ( exp(i p Ipp) _ exp(i p 1 l PO ) \ 

2ord£ ^ |det(M PO - I )) 1 / 2 \ ' l P ' l P' / 


which is nothing but the average over all contributions (16) with different p £ £. For the 
definition of M PO see (19), x PO is the corresponding phase factor. 

Let us put the trace formula into a slightly more general form by introducing test functions 
h : C —> C with the properties 

(i) h is even, i.e., h(p) = h(—p) } 

(ii) h is holomorphic in the strip |Imp| < <r, <r > 1, 

(iii) h(p) = 0(|p| -3-5 ) for 8 > 0 as \p\ —>■ oo. 

We multiply (14) by ph(p)/ 7ri and integrate over p from —oo + icr to oo + icr to obtain the 
general trace formula for co-compact hyperbolic lattices in jj 3 , 


(25) 


J2 Mp~) = - 


vol <^» /,"(») 


n=l 


2tt 


+ £ x(p) Ar6 g^^ J dqq h(q) coth(TT^) 
Mr 0 


+ £ 


x(p) 


{p} r 2ord£r(/?)(l - cos, 

ellipt. 


MO) 


+ £ 


x(p) 


+ £ 


Wr 2oTd£ r (p) 

x(p) 


' i lp \ sintL[(7T - 4> p )q\ 

^ k '^ _T "T7- 5 • / 

sinh(7rg) sin (p p 


inv. ellipt. 

O<0<7T 


q 


Wr 2ord MM / dq h ^ sinh(7T 9 ) 

inv. ellipt. 

(f) = 7T 


+ £ 


xb ) 1 


TO 


2 ord£r(r) (cosh l T — cos <f> T ) 

hyperbol. 

X(r) l 


h(l T 


l T 0 


2ord^r('?') sinh/ T 

inv. hyperbol. 


h(l r 
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h is the Fourier transform of h: 

h(x) 

and h" is its second derivative. 

We will illustrate the trace formula in case of a lattice r + whose fundamental cell is the 
pentahedron shown in figure 1. (The index + will become clear below.) r + is generated by 
three rotations: a half turn pbc through the axis BC identifying face A'BC with ABC , a 
Upturn through DB identifying face DA'B with DAB , and a p-turn through DC identifying 
face CD A' with C DA. The eigenfunctions ip(x) of the Hamiltonian —A on L 2 (f^ 3 /T + , 1) satisfy 
the periodicity conditions 

(26) Tp(g(x)) = Ip(x), 

for all g £ T + . As the pentahedron is symmetric with respect to the BCD plane, the energy 
spectrum decomposes into two subspectra, belonging to eigenfunctions that are symmetric and 
anti-symmetric with respect to a reflection g A at BCD : 

(27) ij){g A {x)) = ±ij){x). 

Condition (26) and (27) can be combined to 

(28) 

for all g in the group T = T + UT + gp where x(d) = 1, if <7 is contained in T + , and x(d) = il, if 
g is in Y + g A - The fundamental cell of the lattice T is the tetrahedron T 8 = ABCD. Equation 

(28) induces Neumann or Dirichlet boundary conditions, respectively. 

It should be noted that the tetrahedron T 8 is symmetric with respect to a half turn p v with 
axis running through the midpoints of AB and CD. Therefore again our energy spectrum 
decomposes into two further symmetry classes. Anyway, as the main target of this work is to 
demonstrate the connection between geometry and energy spectra, we would like to keep the 
geometry as simple as possible. The total decomposition of our spectrum will be postponed to 
a later publication, where we focus on spectral properties. 

The next section provides some facts about lattices generated by reflections, especially 
tetrahedral lattices. 



Ill Hyperbolic Tetrahedra 


Let T be a reflection group, i.e., a lattice generated by the reflections gi, ... ,g n at the faces 
of a polyhedron. The discreteness of T implies the following conditions on the gi [6]: Firstly 
each pair of generators should generate a discrete subgroup (<?«•, <?j) of 0(2). Remember that 
the product gtgj will be a rotation through an angle 2i r/A, k £ N, that is twice the dihedral 
angle at which both mirror planes meet. The Coxeter diagram of such a simplex looks like 



A vertex Q describes a plane mirror, the graph an edge of order k. For A: = 2,3,4,... we draw 
alternatively 
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plane of 
sy mm etry 



Figure 1: 

T 8 . 


The pentahedron ABC DA ', which can be decomposed into two copies of tetrahedron 
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o , o 
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Let us turn to the corner points, at which at least three mirrors meet. The subgroup of T that 
leaves a corner point invariant has to be a discrete subgroup of 0(3). Imagine the subgroup 
acts on the two-sphere, then its fundamental cell should be a spherical polygon, whose corner 
angles correspond to the dihedral angles between the mirrors. For example suppose a corner 
surrounded by three mirrors, you will obtain a spherical triangle. The sum of angles in a 
spherical triangle is greater than 7r, and we get the condition 

(29) -—|- y -|->1? l,l,m £ N - {!}, 

k l m 

i.e., we can admit corners of the type 

777 

o o-o , o-o-o , 

0-0 0 , or 0-0 0 • 

Combine these corners to, e.g., a tetrahedron, which can be embedded into spherical, Euclidean 
or hyperbolic space. There are exactly nine compact hyperbolic tetrahedra T) [7], 


o-o 



o-o 

t 4 = 

o o 

o-o 

t 7 = 

o-o 


0 ^= 0 
t 2 = 

o o 

o-o 

t 5 = 

o-o 

o-o 

Ts = 

o-o 


o-o 

Ts = 

o o 

o-o 

Te = 

o-o 

o-o 

t 9 = 

o-o 


out of which only Tg is not arithmetic [8]. In addition we can construct non-compact tetrahedra 
having a cusp at infinity , instead of a corner. For a cusp condition (29) should be replaced by 

11 1 
k l m 

Anyway, let us stick to the compact case, and have a closer look at the tetrahedron Tg as well 
as its reflection group, which we will denote by T. 

Tetrahedron Tg is shown in figure 1 with corner points A, B } ( 7 , D and dihedral angles 

ABC = 7t/2, AC A = 7t/3, AAB = tt/4, 
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ZD A = 7r/2, ZDB = 7r/3, ZDC = tt/5. 

The volume of Tg is Vol(T 8 ) ss 0.358653. We cut open the tetrahedron at the even edges BC, 
AB and DA , and unfold the surface into the BCD- plane. We then obtain a hexagon (see figure 
2) with corner angles 

7t/2, 7t/2, 7t/4, 7t/2, 7t/2, 7t/4, 

which is a fundamental cell of a two-dimensional lattice, generated by the reflections (see [9]) 

hi = gcgA 9 D gc ( gc 9 A 9 D )~ 1 
h 2 = Qd 

h 3 = ( gB9A) 2 gD9B 9a [( gB9A) 2 gD9Br 1 
hi = ( 9b9a) 2 9d9b9c 9d [{9b9a) 2 gogsgcT 1 
h 5 = (9 b9a) 2 9c ( 9b9a )~ 2 
K = 9c9a 9b ( 9c9a)~ 

where 9a,9b,9 c, 9d are the reflections at the face opposite to point A, B ,C , D , respectively. 

Following the Gaufi-Bonnet theorem, the surface area equals 37t/2. As each face of Tg can 
be rotated into the BCD- plane by an element of T, each reflection gB, 9 c, 9 D is T-conjugate to 
9a- We have only one conjugacy class of plane reflections, hence one can only choose identical 
boundary conditions on all faces of the surface, either Neumann or Dirichlet type. 

Next consider elliptic conjugacy classes. First of all, each rotation of Y is conjugate to its 
inverse, as 

(31) 9 i 1 {9 l 9i)9 l = 9j9t = (fiWj) -1 , for i,j = A, B , C, D. 

Rotations through distinct angles cannot be conjugate. Let us check if the half turns through 
BC and DA are T-conjugate, or - which is equivalent - if the axes BC and DA are T-equivalent. 
Suppose they are, then there must be an element g £ Y transforming B into A and C into 
D. g maps A and D onto points that are equivalent to B and (7, respectively. Therefore g is 
T-conjugate to the symmetry p v exchanging A with B and C with D, which is not contained 
in T. Hence g (ji Y - contradiction! Similar arguments show the non-equivalence of the turns 
through CA and DB. Table 1 gives an overview over all elliptic conjugacy classes. It should 
be noted that for (f> p ^ tt we have ord£r = 47T /<f> Po , where p 0 denotes the smallest rotation such 
that p = /Jq, with a suitable k £ N. For <f) p = 7r, we have ord£r = 8 tt/(/) P o , if there is another 
half turn in Y with axis orthogonal to the axis of p. If not, then ord£r = 47T /(f) po . In our case 
we get ord£r = 87T /cf> Po for all half turns, just have a look at the axes of the corresponding half 
turns in the hexagon. 

Let us turn to inverse elliptic conjugacy classes. The corner points are not T-equivalent to 
one another, otherwise there would be a transformation conjugate to the symmetry p v (similar 
arguments as above). Therefore inverse elliptic elements leaving distinct corner points invariant 
are not conjugate. 

The dihedral angles at a corner of our tetrahedron Tg are of the form (7r/2, 7 t/3, tt /n), where 
n = 4 for corners A and B } n = 5 for corners C and D. Let £ n be the subgroup of all elements 
of T leaving a particular corner point invariant, it is generated by 

(32) £ n = (gi,gj,gk \ (fiWj) 2 = {9 3 9kf = ( 9k9i) n = 1). 

The order of £4 equals 48, the order of £ 5 equals 120. 


(30) 
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Figure 2: Unfolded surface of tetrahedron Tg. The resulting hexagon with corner angles 
7 t/2, 7t/2, tt/4, 7t/2, 7t/2, tt/ 4 is presented in the Poincare disc 3D = {~ £ C | |~| < 1}, with 
metric ds 2 = 4(1 — \z\ 2 )~ 2 (dz dz). In this model of two-dimensional hyperbolic space, the 
geodesics are straight lines and circles perpendicular to the unit circle. 


axis 

representative p 

phase fp 

length l To 

ord£r ( P ) 

BC 

9d9a 

7r 

2d(B,C) 

8 

CA 

9d9b 

2 7T / 3 

2 d{C,A) 

6 

AB 

9d9c 

tt/2 

2 d{A,B) 

8 

AB 

(9D9c) 2 

t r 

2d{A,B) 

16 

DA 

9Bgc 

7 r 

2 d{D,A) 

8 

DB 

gc 9 a 

2 7T / 3 

2d(D,B) 

6 

DC 

9a9b 

2 7T / 5 

2 d{D,C) 

10 

DC 

(. 9a9b ? 

47t/5 

2 d{D,C) 

10 


Table 1: Elliptic conjugac.y classes. The lengths of edges can be calculated with the second 
laws of cosines of hyperbolic and spherical geometry. We have d(B,C) = d(D,A) ~ 2.273112, 
d{C,A) = d{D, B) « 2.132751, d{A,B) « 1.487102, and d{D,C) « 2.224036. 
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corner 

representative p 

phase (f) p 

ord£r(/tf) 

A 

gBgcgo 

7t/3 

6 

A 

(. gsgcgo) 3 

7 r 

48 

A 

gsigcgD) 2 

7t/2 

8 

B 

gcgogA 

7t/3 

6 

B 

( gcgogA ) 3 

7 r 

48 

B 

( gcgofgA 

7t/2 

8 

C 

gogAgs 

7t/5 

10 

C 

{gogAgs ) 3 

37t/5 

10 

c 

\gogAgBf 

7 r 

120 

c 

goigAgB) 2 

7t/3 

6 

D 

gAgBgc 

7t/5 

10 

D 

(. dAgBgc ) 3 

37t/5 

10 

D 

( gAgBgcf 

7 r 

120 

D 

C 9 AgB) 2 gc 

7t/3 

6 


Table 2: Inverse elliptic conjugacy classes 


We will now show that in the case n = 4 there are exactly three Y-conjugacy classes of 
inverse elliptic elements, and in the case n = 5 there are exactly four classes, compare table 2. 

Because of symmetry reasons we only have to look at the corners A and C. Firstly it is easy 
to see that there is a one-to-one correspondence between r-conjugacy classes of inverse elliptic 
elements of Y belonging to corner point A and £ 4 -conjugacy classes of inverse elliptic elements 
of £4. In the same way we have a one-to-one correspondence between r-conjugacy classes of 
inverse elliptic elements of Y belonging to corner point C and ^-conjugacy classes of inverse 
elliptic elements of £ 5 . (This is not necessarily true, e.g., for classes of plane reflections.) 

Consider n = 4. To be sure that we have not missed an inverse elliptic conjugacy class 
we just count the elements in each class to see if the total number yields 24 (=number of 
orientation reversing elements in £4) minus the the number of elements in conjugacy classes 
of plane reflections. There are exactly two £ 4 -conjugacy classes {gi}s i and {g 1 }s i of plane 
reflections, containing 3 and 6 elements, respectively: Remember that the number of elements 
in a conjugacy class {g}s i equals the index ord£ 4 /ordC,s 4 (g) of the centralizer Cs i (g) in £ 4 , 
which is 48/16 = 3 (resp. 48/8 = 6 ) in our case. 16 (resp. 8 ) is the order of the centralizer 
and corresponds to the angle 7 r /4 (resp. 7 r/ 2 ) at corner A 2 (resp. A\) of our hexagon, as there 
are exactly 16 (resp. 8 ) elements of £4 that commute with (resp. gj), namely the identity, 3 
(resp. 1) rotations with axis perpendicular to the reflection plane, 4 (resp. 2) half turns with 
axis in the reflection plane, and all products of those elements with the considered reflection. 

The elements gig 3 gk } (gig^gk ) 3 and g^gugi ) 2 are not conjugate to one another, the orders 
of their centralizers are 6 , 48 and 8 . Hence their conjugacy classes contain 8 , 1, 6 elements, 
respectively. Now (3 + 6 ) + (8 + 1 + 6 ) = 24, hence we have not missed any conjugacy class. 

In the case n = 5 we have one conjugacy class of plane reflections, which contains 120/8 = 15 
elements. The centralizers of gig 3 gk } (gigjgk) 3 , (<h'<7j<7fc ) 5 and g^gugi ) 2 have orders 10, 10, 120 
and 6 . (15) + (12 + 12 + 1 + 20) = 60, which is exactly the number of orientation reversing 
elements in £ 5 . This completes the proof. 
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To give a first impression of the use of our trace formula (25) we will derive an average law 
for the spectral staircase 

OO 

(33) A f{E) = N+ J2 0(P~Pn), P = ^E^ I, E> 1, 

ra=JV+l 

using only the geometric data discussed so far. (N denotes the number of “small” eigenvalues, 
see section II.) We introduce the smeared spectral staircase 

OO 

(34) M‘(E) = '£h<(p,), 

n=l 


with the function 
(35) 


h e (q) = 


2 L 


ert |IVliUerf^ + ' ? 


e > 0, 


satisfying conditions i-iii of section II. We split the trace formula expression of Af t {E) into its 
average part 


Tie) 


(36) 


67r 2 

1 


+ ± E xM Area(P,) J dq qh,{q) coth (wq) 

Mr 0 


1 


X(p) 


+ 27T ^ ord£(p) (1 — cos cf> p ) 

ellipt. 


p 


1 


x(p) 


+ 2 Mr ord ^r(d) l 

x(p) ] 


dq h,(q) ” Mq] 


sinh(7rg) sin 1 


inv. ellipt. 

O<0<7T 


1 


and its fluctuating part 

(37) A'l(E):=2- E 


+ - V 

2 j” ord£r(p) J 

fpfr vr/ 0 

inv. ellipt. 

(f) = 7T 


X{t) It 


dq h t (q) — 


sm 


h(Trq) 


sin pl 7 


2 tt ord£(r) (cosh l T — cos cf> T ) l 7 

hyperbol. 

1 — X(r)l 


/ c 7 2 

exp --/ T 


+ 


£ 


TO 


sinp/ T 


;2 

ex p -74 


2tt f—f ord£(r) sinh l T l T \ 4 

Mr v 7 

inv. hyperbol. 


Performing the limit e —>■ 0 + for ’W e (E) and neglecting exponentially small corrections we obtain 
a Weyl series in p: 

(38) AT( E ) = c 3 p 3 + c 2 p 2 + cip + c 0 + 0(e _7rp/5 ), p —>■ 00 , 
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where 


c 3 


VoI(Jt) 

Qtt 2 


c 2 


I] x(p) Area('Pp), 

Mr 


inv, 


l 


Cl = 


E 


x(p ) 1 


TO 


2ir j vw ord £(d) (1 - cos <j> p )' 


Mr 

ellipt. 


C 0 


l 

967r 


E x(/») Area(P p ) + 

Mr 


inv, 


}_ _ x(p) _ 

4 {p} r °rd5 r (p)(l — cos cf> p )' 

inv. ellipt. 


This result holds for any co-compact lattice in T) 3 . Inserting the geometric data of Tg and choos¬ 
ing Dirichlet boundary conditions (i.e., x(di) = — 1 for all generators), we get c 3 ss 0.006056, 
c 2 = —3/32, ci = d(B } C)/ 8 ir + 2d(C\ A)/9 t: + 5d(A, B)/32 t: + d(D,C)/5n ~ 0.456854, 
c 0 = —23/32, see figure 7. 


IV The length spectrum of periodic orbits 

The above derivation of the trace formula has shown the intimate relation between the classical 
periodic orbits and the conjugacy classes of the reflection group T of the polyhedron. This 
suggests that the length spectrum of the classical periodic orbits can be computed by the 
conjugacy classes since a geometric method would be a too tantalizing effort in three dimensions. 
To this aim we use the generator product method, which has been used, e. g., in the computation 
of length spectra of hyperbolic octagons, which are surfaces being generated by Fuchsian groups 
[10]. This method generates all group elements g £ T which can be represented by a product 
of the generators 9 b, 9 c and go having at most N factors. Since the periodic orbits are 
directly connected with the conjugacy classes and not with the group elements, one has to 
count only group elements belonging to distinct conjugacy classes. 

From definition (15) of conjugacy classes, it follows immediately that all cyclic permutations 
of a product belong to the same conjugacy class. Thus only the generator products are taken 
into account which are not cyclically equivalent to each other. If the reflection group V was a 
free group this method would already yield all conjugacy classes. Unfortunately there are 10 
identities between the generators by which the products can be transformed and which in turn 
have to be compared with respect to cyclic equivalence. For the considered tetrahedron these 
relations are 

( 39 ) g\ = g 2 B = g 2 c = g 2 D = 1 

and 

(40) ( go 9a ) 2 = ( 9d9b ) 3 = {gD9c ) 4 = (9b 9c) 2 = (gc 9a ) 3 = (,9a 9b ) 5 = 1 

where 1 denotes the unit element. The computer program, which computes the length spec¬ 
trum, uses symbolic algebra with four letters ’1’ up to ’4’ corresponding to the four generators 
9 a,9b,9 c,9 d- With this choice of letters a map from the generator product onto integers is 
obtained by interpreting the word as a number. Then to every conjugacy class belongs a “min¬ 
imal word” having the smallest number. The program starts with generating the words from 
the lowest possible number up to words with at most 20 letters. It then tries to reduce the 
word towards a word corresponding to a smaller number. If this is possible, the considered 
word belongs to a conjugacy class which has already been taken into account, otherwise the 
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Figure 3: The classical staircase N(l) is shown in comparison with the asymptotic behaviour 
(41) for l < 6. The lower staircase counts only surface orbits. It is displayed together with 
Ei(/). 


periodic orbit belonging to this word is stored. To reduce the word, all combinations of the 
identities (39) and (40) together with cyclic shifts have to be considered. Among the words 
with length 20 there are conjugacy classes with up to 2523 possible equivalent words. This 
contrasts to the earlier computation of the length spectrum of hyperbolic octagons where only 
one identity relation exists which only allows the reduction of very few words. Up to word 
length 20 there were 35 855 periodic orbits. It is worthwhile to note that this algorithm yields 
not only hyperbolic conjugacy classes but also elliptic and parabolic ones (if there are any). 
The only quantity in equation (14) being undetermined until now is ord£r( r)- It is obtained by 
its geometric interpretation to be one for periodic orbits lying in the interior of the fundamental 
cell and to be two for surface orbits which do not lie along the edges of the fundamental cell. 
The order of edge orbits is given by the number of cells around the considered edge. Therefore, 
the computer program has to determine the geometric location of each orbit to get ord£r( r)- 
The classical staircase N(l) counts the number of primitive periodic orbits with length less 
than or equal to l. The asymptotic behaviour of N(l) is given by [11] 

(41) N(l) ~ Ei(rZ) ~ eXp( ; W) , / — > oo, 

rl 

where r denotes the topological entropy, which in the case of finite hyperbolic spaces of constant 
curvature K = — 1 is connected with the dimension D via r = D — 1. 

Since thq topological entropy r together with the mean level density d(E) determines the 
efficiency of periodic-orbit theory [12], a larger topological entropy implies that numerical appli- 
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Figure 4: The staircase counting only the periodic orbits having distinct lengths is shown in 
comparison with the fit Nf Ust (l) = ae bl with a = 0.52286 and b = 1.35668. 

cations of periodic-orbit theory are much more difficult in three dimensions. In figure 3 (upper 
curve) the classical staircase N(l) is shown in comparison with the asymptotic behaviour (41) 
with r = 2 for l < 6. This demonstrates that the; computer program has provided the length 
spectrum up to l = 6 nearly complete. Indeed, the first deviation from the asymptotic be¬ 
haviour occurs at / ~ 5.5, which is the cut-off length used in the evaluation of the trace formula 
in the next section. In addition the staircase counting only surface orbits is also shown. Since 
the surface can be considered as a two-dimensional billiard, the topological entropy of the sur¬ 
face orbits is r = 1. The corresponding staircase is compared with Ei(/). Since surface orbits 
are represented by unusually long words in the code' used, these orbits are not so completely 
computed as non-surface orbits. Thus their staircase lies somewhat below the curve belonging 
to Ei(/). This figure gives an impression of the wealth of periodic orbits in three dimensions in 
comparison with two dimensions. 

Since the tetrahedron Tg is not arithmetic, one expects a generic length spectrum of periodic 
orbits, where the multiplicities of the lengths are determined only by the symmetries of the 
classical system. As the equations of motion are symmetric under time reversal, the multiplicity 
of a length should be one, two or four depending on whether the periodic orbit possesses a back 
traversal or not, and whether it is invariant with respect to the symmetry p K . It is therefore 
very surprising to observe a length spectrum with exponentially increasing multiplicities as a 
function of length. This immediately reminds us of the eight relatives of the tetrahedron Tg, 
which are all arithmetic and thus possess an exponentially degenerated length spectrum with 
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Figure 5: The strongly fluctuating multiplicities are displayed as dots. A local average of the 
multiplicities over A l = 0.1 (diamonds) suppresses the fluctuations and is consistent with the 
mean behaviour following from the fit of Ndi st (l) (full curve). 


a mean multiplicity 

(42) (g(l)) ~ c r gX1 ^ , l -> oo, 

being a consequence of the slow increase of the- 'number of distinct lengths 

(43) N dist {l) ~ of 1 exp(/). 


For quaternion groups, whose trace field is invariant under complex conjugation, the constant 
is given by [13] 


(44) 


r 2 d ~ 3 i r 


where cl is the degree of the trace field I\ of Y over the rational numbers Q, and D a the 
discriminant of the minimal ideal a of I\ that contains all traces of T. 

Indeed the trace fields of all arithmetic reflection groups are invariant under complex con¬ 
jugation, therefore they are commensurable to quaternion groups of the type described above, 
and should satisfy (42), with a different constant, however. 

The exponential behaviour of multiplicities of arithmetic manifolds was detected in the 
two-dimensional case first for the regular octagon [10], and later predicted for more general 
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Figure 6: The length spacing distribution P{s) of the unfolded length spectrum is shown in 
comparison with the Poisson distribution -Pp 0 isson(s) = e _s - The histogram contains all spacings 
up to length l = 5.5. 


arithmetic surfaces [14, 15, 16], where 


(g{!)) ~ c r 


cp(//2) 


l —> oo, 


with a constant explicitly known for quaternion groups and some special cases. For details 
see [16, 17]. We would like to note that the relation between the mean multiplicities for 
commensurable groups r a , T& given in [16, 17] is not valid in general. 

The spectral statistics of a chaotic system depend very sensibly on the behaviour of the 
multiplicities in the length spectrum. Quantum energy spectra of arithmetic surfaces even show 
a Poisson-like behaviour, which is typical for integrable systems and absolutely unexpected for 
chaotic systems [14, 15, 17]. Poisson-like statistics are also predicted for three-dimensional 
arithmetic systems [13]. Besides that, there are-.no other chaotic systems, for which there 
exist more rigorously proven results concerning quantum spectral properties, see [18, 19] for 
reference. We have decided to look at the more generic tetrahedron T 8 , which is - as stated 
above - the only non-arithmetic one. 

21 

Since the expectation for a generic system is N& ist(0 ~ , it is interesting to determine 

the kind of exponential growth. In figure 4 N& i st (/) is shown in comparison with a fit N^ ist (l) = 
ae bl with a = 0.52286 and b = 1.35668. A fit function with the asymptotic behaviour of 
Ei(a/) could be excluded. Since a length spectrum of an arithmetic system would have yielded 
6 = 1, l lie multiplicity is smaller for the tetrahedron T 8 but nevertheless exponential. The 
mean multiplicity corresponding to the fit is given by (<//(/)) = e bl . Th& true multiplicities 
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are strongly fluctuating as shown in figure 5. For that reason figure 5 also displays a local 
average over a length interval of A/ = 0.1 which reveals the increase of the multiplicities, 
and also the mean behaviour (<?/(/))• The strong fluctuations are reminiscent of an arithmetic 
system like the regular octagon for which the fluctuations of the multiplicities are shown in [10]. 
There remains to answer the crucial question, why the non-arithmetic tetrahedron T 8 displays 
properties similar to arithmetic systems. 

At the end of this section we want to discuss the length spacing distribution P(s). This 
quantity was originally studied in connection with the quantal level spectrum and is known as 
nearest neighbour level spacing distribution. The length spacing distribution was first studied 
in [20] for the hyperbola billiard which possesses a length spectrum being generic in the sense 
that the multiplicities are in accordance with the symmetries. In [20] it was shown that P(s) 
displays a Poisson distribution. In figure 6 the length spacing is shown for the tetrahedron T 8 
where all length spacings up to length / = 5.5 are included. The distribution roughly agrees 
with a Poisson distribution, and one observes a length clustering since the distribution has 
its maximum near s = 0. Thus in this three-dimensional billiard the length spacing agrees 
empirically with the one of the hyperbola billiard despite its anomalous behaviour with respect 
to the multiplicities. 

V The spectral staircase as a quantization tool 

In this section we would like to demonstrate as an application of the trace formula (25) the 
computation of the spectral staircase Af(E) again for the tetrahedron T 8 with Dirichlet bound¬ 
ary conditions. The main motivation for computing the spectral staircase Af(E) arises from 
the fact that it provides a very useful quantization rule. 

In order to compare the results obtained from classical quantities with the quantum mechan¬ 
ical ones, the quantal level spectrum has to be computed. To this aim the boundary element 
method is employed. Define the following differential operator C x := — — E } x £ IR 3 , then 

the Schrodinger equation (4) reads 

(46) C x i>(x) = 0 . 

The free Green function given in equation (10) satisfies 

(47) C x G(x,y;E) = S(x - y) , 

with S(x) being the three-dimensional Dirac delta distribution. Let S be the surface of the 
polyhedron, then a solution ip(x) of the Schrodinger equation can be represented as a surface 
integral for x (ji E 

(48) tjj(x) = dY< y G(x,y;E) <%) , 

as is verified by applying C x on equation (48). The function $(j/) with y £ E has to be 
determined, such that ip(x) satisfies the boundary conditions for x £ E. Expanding $ in 
terms of simple ansatz functions, a set of linear equations is obtained from (48) in terms of 
the expansion coefficients c 4 - of $. The Dirichlet boundary conditions are imposed by setting 
ip(x) = 0 for x G E. The resulting system of equations Mec = 0 has non-trivial solutions only 
if the determinant of the energy dependent matrix Me vanishes, i.e., if E corresponds to a 
quantal level E n . Since the matrix Me is nearly singular, a singular value decomposition is 


21 




Figure 7: The spectral staircase Af{E) is shown in comparison with the average part W(E) 
(dotted curve) and the full periodic-orbit theory (full curve), where in the latter all periodic 
orbits with length l < 5.5 are taken into account. 

applied to Me- Then the zeros of the singular values betray the locations of the quantal levels. 
To obtain the quant al levels with respect to the two symmetry classes of the tetrahedron Tg 
the Green function G ,:t (.r, y; E) = G(x, y; E) ± G(x, p K (y); E) is used. All quant al levels in the 
energy interval E £ [0, 500] have been computed. For both classes 31 quant al levels are found. 

Let us now turn to the computation of Af{E) in terms of the classical quantities. The 
asymptotic behaviour E —> oo is already determined without the hyperbolic conjugacy classes 
and has been derived in section III, see equation (36). The fluctuating part Afa{E), defined in 
equation (37), due to hyperbolic conjugacy classes, i. e., due to the periodic orbits, describes the 
deviations of Af{E) from the average part (37). The series over the hyperbolic conjugacy classes 
in (37) are absolutely convergent for e > 0. However, to extract from the periodic orbits as 
much information as possible, the smallest possible smoothing parameter e should be used. For 
e = 0 the series are not absolutely convergent, however, they may be conditionally convergent. 
The numerical evaluation of the periodic-orbit sums up l = 5.5 indicates that the characters \ 
attached to the periodic orbits are distributed in a sufficiently random way such that both sums 
are conditionally convergent. In figure 7 the spectral staircase Af{E) is evaluated for e = 0 using 
the length spectrum up to l = 5.5 (full curve). In addition, the average part (E) is shown 
(dotted curve) also for e = 0 which differs from the Weyl series (38) only by exponentially small 
terms. Both curves can be compared with the “true” spectral staircase using the quant al levels 
obtained by the boundary element method. Surprisingly, the average part W(E) describes the 
mean behaviour of Af{E) down to the lowest state, an observation made also in many other 
billiard systems. The contributions of the periodic orbits with length l < 5.5 already conspire 
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0 50 100 150 200 250 E 300 

Figure 8: The quantization function cos( 7 rJ\f (E )) (full curve) is shown together with the quanta! 
levels shown as diamonds and squares for positive and negative symmetry class, respectively. 
The dotted curve belongs to cos(ttW(E)) 

to yield the first step where the curve lies around one in the range E ~ 100 to E ~ 140. Steps 
corresponding to higher excited states are not reproduced. The contributions of periodic orbits 
with lengths l > 5.5 are required to generate the higher steps. However, a close look at figure 
7 reveals that for not too highly excited levels the full curve crosses the steps at almost half 
height, which is not the case for the average part W(E). This behaviour is important for the 
application of the spectral staircase as a quantization tool. 

In [21, 22] an approximation of the spectral staircase using only geometric quantities has 
been employed to quantize three different billiard systems by using the quantization condition 

(49) cos(7r M{E) ) = 0 . 

The name quantization condition is justified since only classical quantities enter the equations 
(36) and (37). In figure 8 the function cos( -kN{E) ) is shown using the approximated staircase 
computed with the length spectrum up to l = 5.5 (full curve), and it is seen that its zeros agree 
very well with the first quanta! levels indicated by diamonds and squares for the positive and 
negative /^-symmetry class, respectively. It may be surprising that the levels are resolved so 
well by keeping in mind that the staircase itself is only generated for the first level, but, as 
mentioned above, this quantization condition already works fine if the steps are crossed at half 
height. The quantization obtained using only the average part W(E) is shown in figure 8 as a 
dotted curve. One observes, that all zeros correspond to a level being a consequence of using 
the average part W(E). Furthermore, the improvement due to the contributions of periodic 
orbits with length up to l = 5.5 is impressive. 


23 



VI Summary 

This paper is devoted to the study of strongly chaotic compact systems embedded in the three- 
dimensional hyperbolic space having constant negative curvature. Special emphasis lies on 
the relation between the classical and the quantum mechanical aspects being exemplified by a 
trace formula which allows to express a sum over quantal levels in terms of purely geometric 
quantities. In the case of two-dimensional systems it was very valuable to have the exact 
Selberg trace formula for various comparisons with Gutzwiller’s semiclassical periodic-orbit 
theory [2]. It is thus highly desirable to have exact trace formulae for analogous comparisons 
in three dimensions. The first step in this direction has been undertaken in [23] where the 
trace of the regularized resolvent has been derived for compact systems which tessellate the 
three-dimensional hyperbolic space corresponding to billiard systems with periodic boundary 
conditions. The derivation of the trace formula is based on the lattice group which realizes 
the tessellation of the hyperbolic space in terms of the fundamental cell. In [23] the group 
contains only orientation preserving elements. The trace of the resolvent for the more general 
case including orientation reversing elements is derived in this paper. The trace of the resolvent 
(14) is generalized to the trace formula (25) applicable to a vast class of test functions. The 
inclusion of orientation reversing elements has the advantage that in addition to the periodic 
boundary conditions also Dirichlet and Neumann boundary conditions are covered by this trace 
formula. An exact trace formula has thus been found for all compact billiard systems which 
tessellate the hyperbolic space. 

There are three-dimensional models which have been proposed in [1] for a study of quantum 
chaos. However, they are too complicated for a first application of the trace formula. The 
simplest three-dimensional systems are provided by tetrahedra out of which nine are possible in 
hyperbolic space. Eight of these are arithmetic and thus lead to non-generic quantum systems. 
Therefore, we have chosen the only non-arithmetic tetrahedron, called Tg. The conjugacy classes 
of Tg have been discussed in detail and the length spectrum of periodic orbits, connected 
with hyperbolic and inverse hyperbolic conjugacy classes, has been numerically computed. 
The numerical result indicates that the multiplicities of periodic orbits with identical lengths 
increase exponentially with length, a result unexpected for a non-arithmetic system. The 
unusual properties of arithmetic quantum chaotic systems are ascribe to these exponentially 
increasing multiplicities. The computed length spectrum allows an approximate computation 
of the spectral staircase Af(E) which in turn provides a valuable quantization tool, see equation 
(49). In order to compare these results directly with quantum mechanics, the quantal levels up 
to E = 500 for Tg with Dirichlet boundary conditions have been computed using the boundary 
element method. The first quantization of a three-dimensional chaotic system using periodic- 
orbit theory is displayed in figure 8 where the excellent agreement of the zeros of cos(tv Af (E )) 
with the quantal levels is seen. This is especially impressive because the topological entropy of 
three-dimensional systems in hyperbolic space is r = 2 being the highest topological entropy 
ever encountered in numerical applications of trace formulae of chaotic systems. 


Appendix: Proof of the trace formula 

Elstrodt, Grunewald and Mennicke [23] have derived a trace formula for co-compact Kleinian 
groups, i.e., co-compact lattices of Iso + T) 3 ~ PSL(2,C). To extend their result to general co¬ 
compact lattices T of IsoT) 3 , suppose T is not a Kleinian group. It can be shown that every 
such lattice T contains a Kleinian subgroup of index two, namely the group T + of all orientation 
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preserving elements of T. We have the coset decomposition 

(50) r = r+ u r+m 

A fundamental cell J-p+ can be chosen in such a way that it consists of two copies of the 
fundamental cell of T: 

(51) E r + =f r Uff(f r ). 

As the integration in (12) is independent of a particular choice of the fundamental cell, we 
could as well integrate over cr(J-r), or equivalently over the double cell jF r + such that 

(52) j dp(x) [Gt(®, x\ E) — Gr(x, x\E')\ = — j dp(x) [Gy(x^ x\ E) — Gr(x, x\E')\. 


Ft 


F 


r+ 


For the rest of this derivation suppose Imp > 1, Imp' > 1. We write Gr(x } x] E) as the coherent 
superposition (9). We can carry out the same estimate as in [23] to see that we are permitted 
to exchange sum and integration. We then obtain 

( 53 ) 7jE J d d( x ) x(g) [G(g(x),x;E) - G(g(x),x;E')] 


ge r 




r+ 


= - E 
2 


dg(x) x(g) [G(g(x ), X] E) - G(g(x ), x] E')] 


3GT+ -p 


r+ 


1 


+ h E 


dp(x) x(g) [G(g(x ), X] E) - G(g(x ), x-, E')\. 


ger+c 




r+ 


The sum over orientation preserving elements of T has been worked out in [23]; the result reads 
dg(x) x(g) [G{g{x ), x-, E) - G(g(x ), X] E')] 

Vol(^ r +) 


yer+ y r + 


(54) 


87ri 


■(P -P ) 


- £ 


x(p) K+ 


l l 


M r+ 8ord£ r +(p) (1 - cos <j> p ) \ip i p' t 

ellipt. 


- £ 


X(r) l T + 


exp(ip/ T ) exp(ip'/ T ) 


8 ord£ r + (t) (cosh l T — cos (f> T ) 

hyperbol. 


1 p 


ip' 


Tq 1 " is a hyperbolic element of r + with minimal length l T + that commutes with p or r, respec¬ 
tively. For further explanations see section II, just replace T by r + . 

Let us turn to the second sum over orientation reversing elements. Using Selberg’s trick 
[24] we rewrite (53) as a sum over r + -conjugacy classes {p}r+ of orientation reversing elements 
g G r - T+, where the integration has to be performed over the fundamental cell of the r + - 
centralizer Cp+ (p), 

(55) ^ E / dp(x) x(g) [G(g(x),x;E) - G(g(x), x; E')] 

hdr+ W r+ (g) 


aer-r+ 
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There are three kinds of conjugacy classes we have to deal with: classes of plane reflections, 
of inverse elliptic elements (we distinguish between point reflections and other inverse elliptic 
elements) and of inverse hyperbolic elements. 


Classes of plane reflections. Each plane reflection p is PSL(2, C)-conjugate to the reflection 
lj at the (aq, x 3 )-plane, i.e., there is a g £ PSL(2,C), such that j = gpg~ x . We have 


(56) 


x(p) 


dp(x) \G(p(x), x; E) — G(p(x) } x; E')\ 




c r+ (p) 


x(p) 


dp(x) [G(}(x),x-,E) - G(j(x),x-,E% 




c e(i) 


where 0 := gT + g 1 . It follows from 


j 


a b 
c d 


± 


a b \ . 
c d J ^ 


that all coefficients a,b,c,d are either real or imaginary, hence the centralizer C©(j) is the 
subgroup of all elements contained both in r + and 

PSL(2,E)UPSL(2,E) * _°. 

C©(j) tessellates the (aq, x 3 )-plane, and can therefore be viewed as a two-dimensional lattice, 
which is naturally co-compact. By V p we denote its fundamental cell on the (aq, x 3 )-plane 
(which in general is a hyperbolic polygon). 

The fundamental cell Ec @ ( j) of the centralizer C©(j) in T) 3 has the shape 

(57) E c@ (i) = ( x e ^3 I ( r ffi) (-71-/2, 7t/2)}, 

with cylindric coordinates 



X\ = z, x 2 = rsmtp, x 3 = r cos p>. 


Hence (56) equals 


( 58 ) 


x(p) f f dz dr f dip exp[ip d(j(x), x)] — exp[ip' d(j(x), x)] 
8 tt JJ r 2 J cos 3 p> sinh d(j(x), x) 


where 

d(](x),x) = 2 arsintJ—— = 2 arsinh| tan <p\. 

x 3 

The (z, r)-integration yields the area of V p} the (^-integration can be transformed into a repre¬ 
sentation of the psi function 

T' 

H x ) : = p-l®)- 

We hnally get 


(59) 


Area(T’p) 

XKP) 


[V’( 1 - ip) + V’(-ip) 


V>(1 - ip') 


V’(-ip')]- 
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Classes of point reflections. Choosing suitable coordinates, the point reflection p corre¬ 
sponds to the reflection p at point x = j, i.e., 

p : = ^ _°j j, ^ = gpg' 1 , 9 C PSL(2,C), 

with suitable g £ PSL(2,C). Again let 0 := gT + g~ 1 . 

The centralizer C@(p) consists of all elements of finite order in 0 that leave the point x = j 
invariant. 

To proof this statement consider the equation 


0 j \ ( a b 


-j 0 J \ c d 


a b WO j 


c d \ -j or 


hence the coefficients must fulfill the relations a = id and b = =pc, therefore 


y c d J ^ =p& ±a J 

The lower sign is ruled out as the determinant would be negative. The trace is real and 
contained in the interval (—2,2), as |a| 2 + \b\ 2 = 1, therefore |a| < 1, hence |Rea| < 1, equality 
if and only if a = ±1, 6 = 0. 

Point j is invariant under such transformations, because 

(aj + h){-b] + a) -1 = j. 


C@(p) is finite (as 0 is discrete) and therefore trivially coincides with its maximal finite 
subgroup £@(p). By conjugation the same is true for C r +(p) = £ r +(p). 

Defining 


/(p, <^):= — jdpfx )^r 


exp[i pd(p(x) } x)\ 
sinh d(p(x) } x) 


, Imp > 1, 


where 


_ ( 0 ie 1 ^ 2 \ . 

P - [ ie -ict/2 Q )j> 


(61) i j dp(x) x{p) [G{p{x),x;E] - G(p(x),x;E')] = j-^^-^[I(p,Tr) - I(p,tt)]. 


Evaluating the integral in definition (60) of 7(p, <f>) we get 


/(p, <j>) = 


J_ P(-ip) 
4ip 2 


if 6 = 7T mod 27r 


if ^0 mod*, 

2ip sm cp fTy ip — k 


-*(§)]• 
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Classes of further inverse elliptic elements, p is conjugate to 


P 


0 ie^/ 2 

ie-i^/ 2 0 


j = 9P9 \ 9 £ PSL(2, C), <t> e (0,tt). 


The coefficients of matrices that commute or anti-commute with p must fulhll a = ±d, b = 
Tce 1 ^ = ±ce _1 A As (j) ^ 0,7r, we have b = c = 0, 


a b 
c d 


a 0 
0 a 


Let 


t = 


^lTr/n Q 

0 e~ iv /' n 


be the smallest rotation of that kind, then the centralizer is the finite cyclic group C@(p) = 
(I) = £@(p) and C r +(p ) = (0 = S r +(p), £ = g~Hg- 
We obtain 


(63) 


i J dp(x) x(p) [G(p(x),x;E) - G(p(x),x;E')] = IHp^p) ~ Hp'^p)}- 




c r+ (p) 


Classes of inverse hyperbolic elements. An inverse hyperbolic element r is conjugate to 

( e '/ 2 0 \ 

t = ( q e -i/ 2 J j = grg , g e PSL(2, C), / > 0. 


It follows from 


e 1 ! 2 0 \ . f a b \_/ a b \ / e}! 2 0 \ . 

0 e“P 2 j J \c d J = ± [ c d J [ 0 e~ 1 ! 2 ) J 


that 


a b \ _ f e}'! 2 0 

c d j 0 e~ 1 '/ 2 


or 


d 


// 2 0 W i 0 
0 e“''/ 2 j 1 0 -i 


, V > 0. 


Let Tq 1 " G 0 = gV+g 1 be such an element with smallest occuring length l T +. The centralizer of 
t is generated by that element and the maximal finite subgroup £@(f) of C@(t) : 


(64) 


C®(t) = (ffi + ) X 


(65) £ 0 (h) = |^ 

We integrate over the set 


1 0 
0 1 


or 


£i 


e m = 


1 0 
0 1 


i 0 
0 -i 


[xi,x 2 G E,1 < x 3 < exp (/ T +)} , 
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which in the first case (ord£e(r) = ord£p+(r) = 1) is the fundamental cell of the centralizer, 
in the second case (ord£e(r) = ord£p+(r) = 2) just the double. We obtain 

(66) ^ f dfi(x) x(r) [G(t(x),x;E) - G(t(x), x; E')] 


T, 


c r+ (r ) 


xO 


xp(/ + ) 


oo oo 


87rord£r+(T) J x 

1 —oo —CO 


dx s f f exp[i p d(r (x) } x)] — exp[ip 1 d(r (x) } x)] 

—w / dx i / dx 2 - — — -p ————r- 

3 J J smn d[T[x)i x) 


where 


cosh d(r(x), x) = 1 + 2 


(x\ + re 2 ) sinh 2 //2 + x\ cosh 2 //2 


After having performed the integration with the help of [25] we get a contribution 

/ 67 x _ XX) f exp(i pl T ) _ exp(ip' / T ) \ 

8 ord£ r + (t) sinh / y ip ip 1 J 

Collecting all contributions we get a formula for the trace of the regularized resolvent for 
Imp, Imp' > 1 


CO 


71 = 1 


( 68 ) 


_ Vol(^ r r+) 

87ri 

- I] x(X 

{/ 9 }r+ 


- £ 


(p - p') 

Area(iPp) 

16 7T 

x(p) K+ 


[^(1 - ip) + V’(-ip) ~ ^(1 - V) ~ ^(-i p')\ 


1 1 


{ P } r+ 8 ord ^r+ (p) (1 - cos <j> p ) yip ip', 

ellipt. 

x(p) 


+ £ 


{p } r+ 2 ord£p+ (p) 

inv. ellipt. 


[i(p, <t> P ) ~ HpXp)] 


- £ 


X(r) l T + 


exp(ip/ T ) exp(ip'/ T 


+ 8 ord£ r + (t) (cosh l T — cos cf> T ) y ip 

hyperbol. 

^ X(r) l T + /exp(ip/ T ) exp(ip'/ T 

8ord£p+(r) sinh/ T y ip 

inv. hyperbol. 


ip' 


ip' 


To put our trace formula into the form (14), we will rewrite the sums over T + -conjugacy classes 
as sums over T-conjugacy classes: 


Case 1. Suppose g £ T is a plane reflection. Then 

{fl'lr = {5 , }r+ U {g}r+<r = {5 , }r+, 

as {p}r+ = {pjr+d = {crpcr}r+, i.e., there exists an h £ T + , such that g = hagah~ x . For 
example, choose h = go. 
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Case 2. Suppose g £ T is not a plane reflection and the decomposition 


Mr = Mr+ U {g}r+a = Mr+ U {crga} r + 

is disjoint. Then there does not exist an h £ T + , such that g = hagah~ x = hagi^ha) -1 . 
In this case no element of T + cr commutes with g, therefore Cr+(g) = Cr(g)- Furthermore 
£r(g) = £r+(g ), in particular ord£p(g) = ord£p+(g) = ord£p+ (aga). The last equality holds, 
as £r+(g) = & £r+( cr g (T ) ° and conjugate groups have the same order. 

Let Tq be a hyperbolic element with shortest length commuting with g. In this case Tq~ is 
not only T + -primitive, but also T -primitive. 

We show this by contradiction. Suppose rfi is not T-primitive. Then there exists a r 0 £ T + cr, 
such that = Tq . Conjugation by r 0 yields rj = t 0 Tq rf 1 , hence r 0 commutes with Tq“ - 
contradiction! 

As Tq" is T-primitive, we write r 0 = Tq~. We abbreviate the summands in (68) by 

ord£p+ 

and obtain 

/ggx 2 l T0 S(l g , <t> g ) = Kf S (h-> M + / T +A(/ g , (f)g) 

ord£r(g) ord£p+(g) ord£p+ {crgcr) 

Now S(l g ,<f>g) = S(lg,-<f>g) and, as -j (trg)j = trg, we get 


It follows 

( 70 ) 


fig) — £{hgh ^igi) ~ £{}crgcr-> ficrgcr)- 


2 l TQ S(l g , fig) K+S{}g 1 <f>g) l T +S(lcrgcr, Merger) 

ord£p(g) ord£p+(g) ord£p+ (crgcr) 


Case 3. Suppose g £ T is not a plane reflection and the decomposition 

Mr = Mr+ U Mr+<7 

is not disjoint. Then {g}p+ = {g} r+ CT , i.e., there must be an h £ T + , such that g = ha (/(her) -1 . 
Hence the centralizer Cr(g) contains orientation reversing elements. Cr+(g) is the subgroup of 
all orientation preserving elements. It is of index two in Cr(g)- 

Cr(g) = C r +(g) U C T +{g)a. 

If Cp+((/)(T contains elements of finite order , a can be chosen such that it is itself of finite 
order, 

£r(g) = £r+(g) u £r+(g)&. 

In this case, a hyperbolic element rfi with shortest length that commutes with g is T-primitive. 

Suppose the contrary, i.e., there is a r 0 £ T — T + with rf = rfi . But now t 0 <t is hyperbolic 
having the same length as r 0 , hence half the length of rfi. The latter should have been a 
hyperbolic element with shortest length - contradiction. 
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We obtain 


(71) 


2 l T: S(lg, 


L+S(! g A g ) 


ord£r(<?) ord£ r +(g) 
If Cr+(g)cr contains no elements of finite order , then 

£r{g) = £r+(g)- 


We show there exists a hyperbolic element T\ with shortest length in Cr+G?); which is not T- 
primitive (so there is an element r 0 in Cr(g) with a shorter length): 

Let Tq 1 " be a hyperbolic element with shortest length l T + in Cr+(g) and r 0 an inverse hyper¬ 
bolic element with shortest length l To in Cr+(g)cr. 

l T + cannot be smaller than / To , because otherwise either t 0 To~ £ Cp+ (g)<r or t 0 To~ 1 £ Cp+ (g)<r 
would have a smaller length than r 0 , which already should have had the smallest length. l T + = 

l To is not possible either, because otherwise T 0 Tfi £ Cr+(g)cr or T 0 Tfi 1 £ Cr+(g)cr would be of 
hnite order. Hence l r + = 2 l To . T\ := rf is obviously not T-primitive. Therefore 

/ 72 x 2 l T0 S(l g ,(/) g ) = 

ord^r(5') ord£ r +((/) 

The above arguments allow to rewrite formula (68) in the form (14). 
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